Statistical Sketch based Anomaly Detection and Validation using an Anomaly Database

An anomaly detection procedure based on statistical profiles of sketches of internet traffic is proposed. To validate its statistical performance, measurement campaigns were conducted to collect regular traffic as well as traffic with anomalies, on the Renater network. Anomalies were produced using real-world DDoS tools (tfn2k, trin00). The attacks target different services (ICMP, SYN, UDP, etc.) and aim at occupying the bandwidth. They consists of volume anomalies, however kept at low intensity levels with no noticeable impact on the global traffic. This leads to the production of a documented, controlled and reproducible anomaly database. The traffic traces are analyzed by means of random projections in a small-dimension space (sketches). Each sketch is aggregated over a collection of different time-scales and modeled using non-Gaussian statistics. Anomalies are detected by quantifying the departures of the modeling parameters from those estimated under normal situations. Such deviations are quantified by means of Mean Quadratic Distance or of Kullback-Leibler divergences. The labeled database enables us to study the statistical performance (false negative vs. false positive) of the proposed detection procedures. They are shown to present satisfactory performance, down to alert times of the order of 1 minute.